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TECHNICAL FIELD 



The present invention relates generally to analyzing performance of investment 
professionals, and more particularly, to a system, method, and computer readable medium 
storing computer-executable instructions for evaluating the performance of securities analysts, 
investment analysts and/or other investment related individuals and/or entities. 



BACKGROUND ART 



In the context of securities and other types of investments, obtaining the advice of 
analysts has always been considered a prudent exercise. These analysts possess expertise in 
evaluating investments, such as securities, mutual funds, bonds, and the like. Most specialize in 
specific industries or sectors to allow for more in-depth research. During the course of providing 
investment advice, each analyst may make any number of upward and/or downward forecast 
revisions on securities in their area of expertise. These forecast revisions, or simply revisions, 
are changes in opinion on how much a company is likely to earn per share. 
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When these analysts speak, institutional as well as individual investors listen, driving 
stock prices sharply up or down depending on the nature of the analysts' latest pronouncements. 
As can be imagined, some analysts are more influential than others. For instance, some analysts, 
affectionately named 'axes', are so accurate that their comments carry disproportionate weight. 
While none are infallible, it is important to at least know and consider what professional analysts 
have to say about a particular stock of interest. With access to this information, users can make 
their own assessments and invest accordingly. Accordingly, we have determined that a need 
exists for efficiently disseminating such analyst information to the public. A need also exists for 
a system, method, and computer readable medium containing instructions utilizable for 
10 providing not only information concerning or considering an analyst's revisions, but also 
information pertaining to the reliability and/or accuracy of the revisions. 

* 3 Several prior art techniques have not adequately addressed these needs. For example, 

} j] U.S. Patent 5,132,899 to Fox discloses a stock and cash portfolio development system. As 
H depicted in prior art FIG. 1 of the present invention (FIG. 1 of Fox ), the system of Fox uses data 
|y gathering and processing methodology to produce a system where a list of stocks and a cash 
^ position is generated and purchased for investment and operating accounts. Specifically, the 
*!f system integrates three areas of data: investment performance for investment managers; Federal 
I* Securities and Exchange Commission reports filed quarterly by investment managers; and 
J"? financial characteristics for a number of stocks, to produce a stock portfolio. For example, in 
10 box 1, investment managers are screened to produce a subset for analysis. From there, managers 
are selected (box 2) and sorted into descending order simply according to rates of return on 
investments (box 3). Subsequently, a consultant reviews the results (box 4) and prints the top 
five mangers (box 5). 

Similarly, U.S. Patent 4,566,066 to Towers relates to a securities valuation system. As 
shown in prior art FIG, 2 of the present application (FIG. 1 of Towers ), the system of Towers , 
comprised of components 10-23, produces securities portfolio valuation schedules for multiple 
simultaneous users. In Towers , a customer communicates with the system through terminal 10 
to access and edit accounts in user file 12. By using a CUSIP routine 18 and an AMOUNT 
routine to reflect stock splits and dividends since the last stock pricing, the system of Towers 
30 produces and displays 23 a selected stock portfolio valuation. 
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In U.S. Patent 5,812,987 to Luskin et al. , an investment fund management system 
manages assets in one or more investment funds over a specified period of time. This system, 
comprised of components 31-39, determines a strategic investment mix of assets in a particular 
fund periodically as a function of changing risk. Each fund is managed by manipulating the 
investment mix of the fund in accordance with criteria related to a diminishing length of time to 
a horizon where cash will be withdrawn therefrom. In prior art FIG. 3 of the present invention 
(FIG. 6 of Luskin et al. \ the investment mix is adjusted by first obtaining investor portfolio 
information 35 and market data 36. This data is used to forecast market risks and returns 37, and 

10 to determine the portfolio risk 38. Then, the anticipated cash flow stream 39, the discount 
function 34, and present value of future cash flow 33 are calculated before producing an 

f3 optimized portfolio 32 of assets. 

f U In U.S. Patent 5,761,442 to Barr et aL , a data processing system selects securities and 

constructs an investment portfolio based on a set of artificial neural networks (prior art FIG. 4 of 
*i the present application, FIG. 2 of Barr et al. ). The system comprises components 41-70 and is 
s designed to model and track the performance of each security in a given capital market and 
1 2 output a parameter which is related to the expected risk adjusted return for the security. Each 
IZ artificial neural network is trained using a number of fundamental and price and volume history 
2Q input parameters 10, 20, 30 about the security and the underlying index. The system combines 
r the expected return/appreciation potential data 50 for each security via an optimization process 
60 to construct an investment portfolio which satisfies aggregate statistics. The data processing 
system receives input from the capital market and periodically evaluates the performance of the 
investment portfolio, rebalancing whenever necessary to correct performance degradations 70. 

However while many of these prior art references disclose adequate methods of 
managing investment funds and portfolios, none make any mention of evaluating an analyst's or 
other investment trader's performance. Hence, what is lacking in the prior art is a technique 
directed not only toward an investment fund, but rather a technique for evaluating analysts and 
30 other traders or investment professionals/advisors based on their influence and accuracy. 

Accordingly, a need exists for a system, method, and computer readable medium containing 
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instructions utilizable for not only disseminating information concerning an analyst's or trader's 
actions, but also for evaluating the analyst's or trader's performance. 

In line with the above, a need exists for a system, method, and computer readable 
medium containing instructions utilizable not only for providing raw information and data but 
also for evaluating performance based on returns observed after a particular revision, historical 
consistency, and/or the number of revisions made by the analyst or investing entity. 

Furthermore, a need exists for a system, method, and computer readable medium 
containing instructions utilizable for considering performance based on revisions including a 
change in opinion including an upward or downward change in opinion of the analyst with 
regard to at least one investment. 

A need also exists for a system, method, and computer readable medium containing 
instructions utilizable for adjusting a performance measure to accentuate a number of revisions 
issued by an analyst. 

Yet another need exists for a system, method, and computer readable medium containing 
instructions utilizable for adjusting a performance measure according to a return amount 
adjustment accentuating the measure according to a return amount realized on an investment. 

Still another need exists for a system, method, and computer readable medium containing 
instructions utilizable for adjusting a performance measure according to a small return penalty 
penalizing an analyst for small returns. 

SUMMARY OF THE INVENTION 

It is a feature and an advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for not only disseminating 
information concerning an analyst's or trader's actions, but also for evaluating the analyst's or 
trader's performance. 
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It is another feature and advantage of the present invention to provide a system, method, 
and computer readable medium containing instructions utilizable not only for providing raw 
information and data but also for evaluating performance based on returns observed after a 
particular revision, historical consistency, and/or the number of revisions made by the analyst or 
investing entity. 

It is another feature and advantage of the present invention to provide a system, method, 
and computer readable medium containing instructions utilizable for considering performance 
based on revisions including a change in opinion including an upward or downward change in 
10 opinion of the analyst with regard to at least one investment. 

It is another feature and advantage of the present invention to provide a system, method, 
• 2 and computer readable medium containing instructions utilizable for adjusting a performance 
jjH measure to accentuate a number of revisions issued by an analyst. 

y It is another feature and advantage of the present invention to provide a system, method, 

Vs? and computer readable medium containing instructions utilizable for adjusting a performance 
Q measure according to a return amount adjustment accentuating the measure according to a return 
iU amount realized on an investment. 

30 It is another feature and advantage of the present invention to provide a system, method, 

and computer readable medium containing instructions utilizable for adjusting a performance 
measure according to a small return penalty penalizing an analyst for small returns. 

One embodiment of the present invention, particularly applicable to securities analysts, is 
now summarized. 

System/Processing Description 

Using a proprietary calculation, analysts may be given a series of performance scores that rank 
30 them relative to (1) other analysts in the "universe" and (2) other analysts that make predictions 
about a specific company. A total of five scores may be produced for each analyst: 
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Analyst Buy Score - Represents the analyst's Buy Score for all companies he/she covers, 
relative to the Universe of Analysts 

Analyst Sell Score - Represents the analysts Sell Score for all companies he/she covers, relative 
to the Universe of Analysts 

Analyst Company Buy Score - Represents the analyst's Score for a specific Company 
Analyst Company Sell Score - Represents the analyst's Score for a specific Company 
Analyst Overall Score - Represents the analysts overall Score for all companies that he/she 
covers 

10 

All scores will be calculated based on an event - the revision of an analyst's earnings forecast. 
J J; Data Preparation Steps 

W The following must be complete before the Analyst calculations are begun: 

% 1 . Analyst Backend [ABE] -this process supplies the data from any suitable data base for 
* the calculations. 

^ 2. ABE data relating to Brokers may be removed from the dB — in this embodiment we do 

12 not want to calculate Analyst scores for these non-analyst entities 

10 

Analyst Scoring Logic 

The following logic will be used to calculate the scores for all analyst/company combinations in 
the Analyst database with at least 2 earnings forecast revisions that are at least 60 days (3 
calendar months) old. These raw scores will later be ranked to formulate the final Analyst scores: 

Raw Buy Analyst Score - Note, all inputs are based on calculated values related to earnings 
estimate revision changes 

For every analyst_id and company_id combination in the analyst_company„total table with 2 or 
30 more earnings estimate revisions^ calculate a raw scare using the following logic (the logic is 
first summarized and then explained in detail): 
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Step 1 Calculate the average 5 day, 20 day and 60 day returns for earnings revisions [5/20/60 

event returns] for each analyst/company combination (done by ABE) 

Step 2 Calculate the t -statistic [t-stat] values for 5/20/60 event returns in absolute terms 

Step 3 Calculate the degrees of freedom [DF] values for 5/20/60 event returns 

Step 4 Look up probability value using DF and [t stat] for 5/20/60 event returns 

Step 5 Calculate raw score using conditional logic 

Step 6 Calculate adjustment to raw score 

Step 7 Calculate adjusted raw score 

10 Step 8 Calculate second adjustment to raw score and new raw score 

Step 9 Eliminate raw scores with returns greater than 1000% 

s*i Step 10 Sort and rank the raw scores to-produce the final Analyst Scores. 

f y For every analystjd and company_id combination in the analyst_company_total table in the 
*'Z Analyst dB with analyst_company_total. events >= 2, perform the following logic: 



H Step 1 

M The 5 day, 20 day and 60 day return values are calculated by ABE for each analyst/company 
|Q combination: 

Average 5day return = analyst__company_total.avg_5dy_return_buy 
Average 20day return = analyst_companyj;otal.avg_20dy_retum_buy 
Average 60day return =analyst_company_total.avg_60dy_return__buy 

Step 2 

The t-stat needs to be calculated in absolute terms (Le., a non-negative number) for the 5 day, 20 
day and 60 day returns using the following formulas: 

30 The 5/20/60day_abs_tstat values may be rounded to 2 decimal places 
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Before beginning this calculation, we must ensure that we will not be dividing by zero. If the 
standard deviation is zero, then we will set the final raw score to zero and move to the next 
analyst/company combination. 



IF analyst_company„total.sd_5dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy combination 



IF analyst_company_totaLsd_20dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy combination 



IF analyst_company_totaLsd„60dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy combination 



Sday _abs _tstat _buy 



analyst _company _ total uivg _5dy ^return _buy 



analyst _company _ total. sd _5dy __buy 
^analyst _ company _ total. num _5dy _event _buy 



20 Jay _ abs tstat _ buy = 



analyst _ company _ total javg _ 20dy _ return _ buy 



analyst _company _ total. sd _20dy _buy 
^analyst _ company _ total. num _ 2Qdy _ event _ buy 



60day _abs __tstat _buy - 



analyst _ company _ total javg _ 60dy return _ buy 



analyst _company total. sd _ 60dy _buy 
analyst _ company „ total. num _ 60dy „ event _ buy 
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Step 3 

The degrees of freedom are calculated by subtracting 1 from the number of observations for each 
return value: 

5day _ dfjbuy = analyst_company_totaLnum_5dy_event__buy-l 
20day_df_buy = analyst_company_totalnum_20dy_eventjbuy-l 

10 60day_df_buy = analyst_company_totalman_60dy_eventjbuy-l 
Step 4 

q For each return period, look up the probability using the absolute t-stat and DF for each 
|jl respective calculation as inputs to a lookup table: 

y IF 5/20/60day_df_buy > 100, THEN set 5/20/60day_df_buy = 100 

a IF 5/20/60_abs_tstat_buy > 20, THEN set 5/20/60day_abs_tstat_buy = 20 

11 IF 5/20/60day_abs tstat„buy < .01, THEN set 5/20/60day_prob_biiy = 0 
5day__prob_buy = lookup value indicated by using 5day_abs tstat_buy and 5day_df_buy 
20day_prob_buy = lookup value indicated by using 20day_abs_tstat_buy and 20day_df_buy 
60day_prob_buy = lookup value indicated by using 60day_abs „tstat_buy and 60day_df_buy 
StepS 

The conditional raw score may now be calculated. The conditional raw scare will be based on the 
30 average of the conditional raw scores for the 5 day period, 20 day period and for the 60 day 

period, which will then be multiplied by 100. The condition to determine whether the probability 
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is added or subtracted is based on whether the average return is positive or negative, the formula 

is as follows ( cond _ raw _ score _buy): 

j[l - (5 + or ~ Sday _ prob _ buy)] + [1 - (.5 + or - 20day _ prob _ buy)] + [1 - (.5 + or - 60day _ prob _ buy)] I 



5 day portion of calculation 

IF analyst_company_total.avg_5dy_return_buy > 0 

THEN subtract 5day_prob_buy from .5 in numerator 

- This next case means that we have a negative average return for the events and we may add the 
probability 

ELSE add 5day_prob_buy to .5 in numerator 
20day portion of calculation 

IF analyst_company_total_avg_20dy_return buy > 0 
THEN subtract 20day_prob_buy from .5 in numerator 

- This next case means that we have a negative average return for the events and we may add the 
probability 

ELSE add 20day_prob_buy to .5 in numerator 

60day portion of calculation 

IF analyst_company_total.avg„60dy_return__buy > 0 

THEN subtract 60day_prob_buy from .5 in numerator 



3 




Logic: 
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- This next case means that we have a negative average return for the events and we may add the 
probability 

ELSE add 60day_prob_buy to .5 in numerator 
Final step 

Take the average of the three calculated values above and multiply the result by 100 
Step 6 

10 Calculate the adjustment to the conditional raw score using the following formula (the 
adjustment is generally negative): 



adj _ raw _ buy = (-20) * 



1 



num. _ 5dy _ events __ buy + nam 20dy _ events _ buy + num. _ 60dy _ events _ buy 



3 - num_5dy__events_buy s num 20dy_events_buy and num_20dy_events_buy are taken from the 
h t analyst company_total table- 

ri Note, the constant above may be changed for Analysts. 
20 Step 7 

Calculate the raw score by adding the adjustment to the conditional raw score: 
raw_score_buy - (cond_raw_score_buy) + (adj_raw_buy) 

- here we are adding a negative number (adj_raw_sell) ? therefore the adjustment should generally 
be negative-- 

Step 8 

new_raw_score_buy = raw_score_buy + (adj2_ raw_buy) 
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Calculate second adjustment to raw_score_buy, variable name: adj2_raw_buy 
The adj2_raw„buy is comprised of the following 4 components: 

(1) 5 day returns portion of calculation 

IF analyst_company_total.avg_5dy_return_buy > 10 
THEN 5day_adj2_buy =10 

10 - This next case means that we have a 5 day return less or equal than 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube 
12 root — 

f if ELSE 5day_adj2_buy = (100 * analyst_companyJ:otal.avg_5dyjreturn_buy) 1/3 

(2) 20 day returns portion of calculation 

s IF analyst_company_total.avg_20dy_return_buy > 10 
3 THEN 20day_adj2_buy = 10 

|Q - This next case means that we have a 20 day return less than or equal 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube 
root — 

ELSE 20day_ adj2_buy = (100 * analyst_company__total.avg_20dy_return__buy) 1/3 

(3) 60 day returns portion of calculation 

EF analyst_company_totalavg_60dyreturn Jmy > 10 
THEN 60day_adj2_buy =10 

- This next case means that we have a 60 day return less than or equal 1000% and should use the 
30 following formula. Note the formula is raised to the third power which is equivalent to the cube 
root - 
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ELSE 60day_ adj2_buy = (100* analyst_companyJ;otaLavgj50dy_retum„buy) 
(4) Small Returns penalty 

IF analyst_company_total.avg_60dy_return_btiy >0.15 
THEN small_ret_buy_adj = o 

— This next case means that we positive returns less than or equal 15%, so we will impose a 
penalty on the raw_buy_score - 

ELSE small_retj3uy_adj = (100* (0.15 - analyst„company_total.avg_60dy„return_buy)) 
Final step 

Combine the 4 components using the following formula to arrive at the final adj_raw: 

adj2_raw_buy = (5day_buy_adj2 / 3) + (20dayjmy_adj2 / 3) + (60day_buy_adj2 / 3)- 
small_ret_buy_adj 

Step 9 

Final step is to eliminate returns from the final scoring that have 60 day returns greater than 
1000% 

IF analyst_company_total.avg 60dy_return_buy > 10 
THEN filter this raw score in the final scoring process 

Step 10 

Analyst Scoring Logic - Ranked (Final) Score 

The following ranking system will be used to determine the final scores for each of the analyst 
scores: 

Important Note: Only analyst that cover companies within the Analyst Universe should be given 
a final Analyst score in this example. 
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Analyst Buy Score 

Once all of the raw scores have been calculated, we can then rank the raw buy scores by sorting 
the results in descending order and separate this list into 100 groups. The group with the highest 
set of scores will be given the ranking of 100 (signifying the best performance), and the group 
with the lowest set of scores will be given the ranking of 1 (signifying the worst). The analysts 
in each group should then be assigned a score equal to the group that they are in. 

Step 1 Develop list of analysts who have received a raw„buy_score at companies within 
the Analyst Universe 

Step 2 Count the number of analysts in the list created in Step 1 
10 Step 3 Sort raw_score list descending for all analyst id/company_id where there is a score. 

[Note: If a secondary sort is needed, use analyst company_total,avg.60day return descending]. 
%2 Step 4 Use the look up table below to divide the ranked list into seven groups, and then 
»/1 within each group divide the ranked raw-score list into an equal number of observations for each 
^ score within the group (as defined in Table A shown below). 

|y Step 5 Record the score for the associated analyst_id and company_id combinations in the 
l~ group 

U Raw Sett Analyst Score - Note, all inputs are based on calculated values related to earnings 
It estimate revision changes 

fi 

For every analyst id and company-id combination in the analyst company total table with 2 or 
more earnings estimate revisions, calculate a raw score using the following logic (the logic is 
first summarized and then explained in detail): 



Step 1 Calculate the average 6 day, 20 day and 60 day returns for earnings revisions 
[5/20/60 event returns] for each analyst/company combination (done by ABE) 

Step 2 Calculate the t -statistic [t-stat] values for 5/20/60 event returns in absolute terms 

Step 3 Calculate the degrees of freedom EDF] values for 5/20/60 event returns 

Step 4 Look up probability value using DF and [t-stat] for 5/20/60 event returns 

30 Step 5 Calculate raw score using conditional logic 

Step 6 Calculate adjustment to raw score 
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Step 7 Calculate adjusted raw score 

Step 8 Calculate second adjustment to raw score and new raw score 

Step 9 Eliminate raw scores with returns greater than 1000% 

Step 10 Sort and rank the raw scores to produce the final Analyst Sell Scores. 



In this example, for every analyst id and company_id combination in the analyst_company_total 
table in the Analyst dB with analyst_company_total.events >= 2 perform the following logic: 

Step 1 

10 The 5 day, 20 day and 60 day return values are calculated by ABE for each analyst/company 
combination, 

y Average 5day return = analyst_company_total.avg_5dy_return_sell 
fy Average 20day return = analyst_company_total.avg_20dy_return_sell 
Average 60day return = analyst_company_total.avg_60dy_return„sell 

Step 2 

m The t-slat needs to be calculated in absolute terms (i.e., a non-negative number) for the 5 day, 20 
l& day and 60 day returns using the following formulas: 

'f* The 5/20/60day_abs_tstat values may be rounded to 2 decimal places 

Before beginning this calculation, we must ensure that we will not be dividing by zero. If the 
standard deviation is zero, then we will set the final raw score to zero and move to the next 
analyst/company combination. 



IF analyst_company_total.sd_5dy_sells = 0, THEN raw_score = 0 
And start over with the next analyst/company buy combination 



30 IF analyst_company_total.sd_20dy_sells = 0, THEN raw_score sell = 0 
And start over with the next analyst/company buy combination 
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IF analyst_company_total.sd_60dy_sells = 0, THEN raw_score_sell = 0 
And start over with the next analyst/company buy combination 



Sday _abs _tstat _sell = 



analyst _ company _ total &vg_ 5dy _ return _ sell 



analyst _company _totaLsd _5dy _ sell 
^analyst ^company _totaLnum _Sdy _ event __sell 



20day _abs _ tstat _ sell ~ 



analyst _ company _ total uivg _ 20dy _ return _ sell 



analyst ^company _total.sd _20dy _sell 
^analyst _company _totaLnum „ 20dy __event _ sell 



60day _ abs _ tstat _ sell - 



analyst _ company _ total \avg _ 60dy _ return _ sell 



analyst ^company _totaLsd _60dy _sell 
^analyst _company _totaLnum _60dy _event _sell 



Step 3 

The degrees of freedom are simply calculated by subtracting 1 from the number of observations 
for each return value: 

5day_df_sell - analyst _company-total. num_5dy_event_event_sell- 1 
20day_dfjsell - analyst jzom,panyjotalnum_20dyjevent_sell-l 
60day_df_sell — analyst _company_total.nun_60dy_event_sell-l 
Step 4 

For each return period, lookup the probability using the absolute t stat and DF for each respective 
calculation as inputs to a lookup table: 
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IF 5/20/60day_df_sell > 100, THEN set 5/20/60day_df_sell = 100 
IF 5/20/60 abs_tstat_sell > 20, THEN set 5/20/60day„abs_tstat_sell = 20 
IF 5/20/60day_abs_tstat sell < .01. THEN set 5/20/60day_prob_sell = 0 
5day_prob_sell = lookup value indicated by using 5day_abs_tstat_sell and 5day_df_sell 
20day_prob_sell = lookup value indicated by using 20day_abs_tstat„sell and 20day_df_sell 
60day_prob_sell = lookup value indicated by using 60day_abs_tstat_sell and 60day_df_sell 
StepS 

The conditional raw score may now be calculated. The conditional raw score will be based on 
U the average of the conditional raw scores for the 5 day period, 20 day period and for the 60 day 

period, which will then be multiplied by 100. The condition to determine whether the probability 
i is added or subtracted is based on whether the average return is positive or negative, the formula 

* is as follows ( cond _ raw _ score _ sell ) : 

J[l - (.5 + or - 5day _ prob _ sell)] + [1 - (.5 + or - 20day _ prob _ sell)] + [1 - (.5 + or - 60day _ prob _ sell)] 1 

I ^ — ^ — r 10 - 

Logic: (Note: We are now using less than zero, which is opposite of the Buy logic) 

5 day portion of calculation 
IF analyst_company total. avg_5dy_return_sell < 0 
THEN subtract 5day_prob from .5 in numerator 

- This next case means that we have a positive average return for the events and we may add the 
probability 
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ELSE add 5day_prob to .5 in numerator 

20day portion of calculation 

IF analyst company_total.avg_20dy_return_sell < 0 

THEN subtract 20day_prob from .5 in numerator 

- This next case means that we have a positive average return for the events and we need to add 
the probability 

10 

ELSE add 20day_prob to .5 in numerator 

\ 3 60day portion of calculation 
I y IF analyst_company_total.avg_60dy_return sell < 0 
THEN subtract 60day__prob from .5 in numerator 

3 - This next case means that we have a positive average return for the events and we need to add 
*f the probability 

|§ ELSE add 60day_prob to .5 in numerator 
Final step 

Take the average of the three calculated values above and multiply the result by 100 
Step 6 

Calculate the adjustment to the conditional raw score using the following formula (the 
adjustment will always be negative): 
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adj _ raw sell = (-20) * 



num _ Sdy events _ sell + num _ 2Qdy _ events sell + num _ 60dy _ events sell 



- num_5dy_events sell, num_20dy_events_sell and num_60dy„events„sell are all taken from the 
analyst_company_total table- 

Step7 

Calculate the raw sell score by adding the adjustment to the conditional raw score: 
raw_score_sell = (cond_raw_score_sell) + (adj__raw_sell) 

- here we are adding a negative number (adj_raw_sell), therefore the adjustment should generally 
be negative — 

Step 8 

new_raw„score_sell = raw_score_sell + (adj2_raw_sell) 

Calculate second adjustment to raw_score_sell, variable name: adj2_raw_sell 

The adj2 raw is comprised of the following 4 components: 

(1) 5 day returns portion of calculation 

IF analyst company_total.avg_5dy_return_sell > 10 

THEN 5day_sell_adj2 = 10 

- This next case means that we have a 5 day return less or equal than 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube 
root- 



ELSE 5day_sell_adj2 = (100 * analysLcompany_total.avg_5dy_return_sell) 



1/3 
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(2) 20 day returns portion of calculation 

IF analyst_company_total.avg_20dy_return_sell > 10 
THEN 20day_sell„adj2 = 10 

- This next case means that we have a 20 day return less than or equal 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube 
root - 

ELSE 20day_sell_adj2 = (100 * analyst_company„totalavg_20dy_return_sell) 1/3 

(3) 60 day returns portion of calculation 

IF analyst_company__total.avg_60dy_return_sell > 10 
THEN 60day_sell_adj2 = 10 

- This next case means that we have a 60 day return less than or equal 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube 
root - 

ELSE 60day__sell_adj2 = (100 * analyst_company__total.avg_60dy_return_sell) 1/3 
(a) Small Returns penalty 

IF analyst_company_total.avg_60dy_return_sell < -0.08 
THEN smaU jret_seU_adj = 0 

- This next case means that we negate returns less than or equal -8%, so we will impose a penalty 
on the raw _sell_score - 

ELSE smalLret_sell_adj = (100 * (-0.08 - analyst_company_total.avg_60dy_return_sell)) 
Final step 
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Combine the 4 components using the following formula to arrive at the final adj2_raw_sell: 



adj2_raw_sell = (5day„sell_adj2/3) + (20day_sell_adj2/3) + (60day_adj2/3) + small_ret_sell_adj 
Step 9 

Final step is to eliminate returns from the final scoring that have 60 day returns greater than 
1000% 

IF analyst_company_total.avg_60dy_return_sell > 10 
THEN filter this raw score in the final scoring process 

Step 10 

Analyst Scoring Logic - Ranked (Final) Score 

The following ranking system may be used to determine the final scores for each of the analyst 
scores: 

Important Note: Only analysts that cover companies within the Analyst Universe should be given 
a final Analyst score in this example 

Analyst Sell Score 

Once all of the raw sell scores have been calculated, we can then rank the raw sell scores by 
sorting the results in descending order and separate this list into 100 groups. The group with the 
highest set of scores will be given the ranking of 100 (signifying the best sell performance), and 
the group with the lowest set of scores will be given the ranking of 1 (signifying the worst sell 
performance). The analysts in each group should then be assigned a score equal to the group that 
they are in. 

Step 1 Develop list of analysts who have received a raw„sell_score at companies within the 
Analyst Universe 

Step 2 Count the number of analysts in the list created in Step 1 
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Step 3 Sort raw_sell_score list descending for all analyst„id/company_id where there is a 
score. [Note: If a secondary sort is needed, use analyst company_total.avg_60dy_return_sell 
descending]. 

Step 4 Use the look up table shown below to divide the ranked first into seven groups, and 

then within each group divide the ranked raw_sell score list into an equal number of observations 

for each score within the group (as defined in the table shown below). 

Step 5 Record the score for the associated analystjd and company Jd combinations in the 

group 

LookUp Table A 

This look up table provides the information needed to assign the Analyst Buy Score and Analyst 
Sell Score for each analyst 

Using the rankings determined in Step 3 of Analyst Buy Score/Analyst Sell Score procedures 
derive the Analyst Buy Score and Analyst Sell Score split the rankings into the 7 groups in the 
table below. 



Group 


Percentages of Total 
Observations 


Analyst Buy Score 
Calculation 


Analyst Buy Score 


1 


Top 5.0% (rounded 
up to the nearest 
whole number) 


Divide the top 5% 
of total observations 
into 10 groups with 
an equal number of 
observations in each 
group 


91 to 100 


2 


Next 12.5% (i.e. 
read sorted records 
until reach 17.5% of 
the total number o 
individuals from 
Step 1 of the scoring 
procedures) 


Divide the 12.5% of 
total observations 
into 15 groups with 
an equal number of 
observations in each 
group 


76 to 90 
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3 


Next 18.0% (i.e. 
read sorted records 
until reach 35.5% of 
the total number of 
individuals from 
Step 1 of the scoring 
procedures) 


Divide the 18.0% of 
total observations 
into 15 groups with 
an equal number of 
observations in each 
group 


61 to 75 


4 


Next 29.0% (i.e. 
read sorted records 
until reach 64.5% of 
the total number of 
individuals from 
Step 1 of the scoring 
procedures) 


Divide the 29.0% of 
total observations 
into 20 groups with 
an equal number of 
observations in each 
group 


41 to 60 


5 


Next 18.0% (i.e. 
read sorted records 
until reach 82.5% of 
the total number of 
individuals from 
Step 1 of the scoring 
procedures) 


Divide the 18.0% of 
total observations 
into 15 groups with 
an equal number of 
observations in each 
group 


26 to 40 


6 


Next 12.5% (i.e. 
read sorted records 
until reach 95.0% of 
the total number of 
individuals from 
Step 1 of the scoring 
procedures) 


Divide the 12.5% of 
total observations 
into 15 groups with 
an equal number of 
observations in each 
group 


11 to 25 


7 


Last 5.0% (rounded 
up to the nearest 
whole number) (i.e. 
read sorted records 
until reach 100.0% 
of the total number 
of individuals from 
Step 1 of the scoring 
procedures) 


Divide the last 5% 
of total observations 
into 10 groups with 
an equal number of 
observations in each 
group 


1 to 10 



In accordance with the principles of the present invention, an analyst's performance (or 
other investment professional or investment advisor) is evaluated by utilizing information 
pertaining to at least one revision made by the analyst involving at least one investment. This 
evaluation includes determining a conditional performance score indicative of the analyst's 
performance relative to other analysts. The performance score is determined, at least in part, by 
considering an average historical performance of the investment following the revision. In 
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addition, the performance score is also determined using a historical consistency of the analyst's 
performances with respect to revisions or reports involving the investment, and a number of 
revisions, recommendations, or opinions made by the analyst. Then, the performance score may 
be adjusted according to one or more adjustments to generate a final performance score. 

These performance scores may be determined based on analyst recommendations, such as 
upward or downward revisions or a combination of both. In particular, a score based on upward 
revisions, or an upward revision performance score, corresponds to the predictive nature of an 
analyst with respect to predicting or influencing increases in earning. In contrast, a score based 

10 on downward revisions, or a downward revision performance score, corresponds to the 

predictive nature of an analyst with respect to predicting or influencing decreases in earnings. 
Similarly, a combined score or, for example, an average of the two previous scores, corresponds 

u to the overall predictive nature of the analyst. 

''J Hence, the present invention provides a method, system, and computer readable medium 

Id storing computer executable instructions for evaluating an analyst's performance by utilizing 
1°* information pertaining to at least one revision made by the analyst involving at least one 

investment. In one embodiment, the present invention includes determining a conditional 
|^ performance score indicative of the analyst's performance relative to other analysts. This 
iff conditional performance score is determined at least in part by considering an average historical 
M performance of the at least one investment following the at least one revision, a historical 
consistency of the analyst's performances with respect to revisions involving the at least one 
investment, and a number of revisions made by the analyst. In addition, the present invention 
includes adjusting the conditional performance score of the analyst according to a return amount 
adjustment to generate a final performance score. 

In another embodiment, the present invention provides a method, system, and computer 
readable medium storing computer executable instructions for evaluating an analyst's 
performance. This embodiment includes utilizing information pertaining to at least one revision 
30 made by the analyst involving at least one investment, including an upward or downward change 
in opinion of the analyst with regard to at least one investment. In addition, the present invention 
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includes calculating a raw conditional performance score indicative of the analyst's performance 
relative to other analysts, said raw conditional performance score determined at least in part by 
considering a measure of variability of the analyst's performance, an average historical 
performance of the at least one investment following the at least one revision, a standard 
deviation of the at least one revision, a number of revisions made by the analyst, and a likelihood 
that the at least one revision will actually produce an expected result. From there, the invention 
includes adjusting said raw conditional performance score according to a return amount 
adjustment to produce a final performance score. 

In another embodiment, the present invention provides a method of evaluating an 
analyst's performance by utilizing information pertaining to at least one revision made by the 
analyst involving at least one investment. In this embodiment, the invention includes calculating 
a raw conditional performance score indicative of the analyst's performance relative to other 
analysts. The raw conditional performance score may be determined at least in part by 
considering a measure of variability of the analyst's performance, an average historical 
performance of the at least one investment following the at least one revision, a standard 
deviation of the at least one revision, a number of revisions made by the analyst, and a likelihood 
that the at least one revision will actually produce an expected result. Subsequently, the 
invention adjusts the raw conditional performance score according to a modifier accentuating the 
number of revisions made by the analyst. In addition, the raw conditional performance score 
may be adjusted according to a small return penalty penalizing the analyst for small returns to 
produce a final performance score. 

In still yet another embodiment, the present invention includes a method, system, and 
computer readable medium storing computer executable instructions for evaluating an analyst's 
performance, and includes utilizing information pertaining to at least one upward or downward 
revision including an upward or downward change in opinion of the analyst with regard to at 
least one investment. This embodiment also includes calculating a performance score indicative 
of the analyst's performance relative to other analysts, said performance score determined at 
least in part by considering a measure of variability of the analyst's performance, an average 
historical performance of the at least one investment following the at least one upward or 
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downward revision, a standard deviation of the at least one upward or downward revision, a 
number of upward or downward revisions made by the analyst, and a likelihood that the at least 
one upward or downward revision will actually produce an expected result. In addition, this 
embodiment includes comparing the analyst's performance score against performance scores of 
other analysts to produce at least one of an upward, downward or combined revision ranking. 

In yet another embodiment, a method of evaluating an analyst's performance by utilizing 
information pertaining to at least one revision made by the analyst involving at least one 
investment is disclosed. In this embodiment, the invention includes determining a first average 
return value (ave. returni), second average return value (ave. return 2 ), and third average return 
value (ave. return 3 ) for the at least one revision. Each of the first, second, and third average 
return values corresponds respectively to an average rate of return on the at least one revision 
involving the investment for a first period of time (timei), a second period of time (time 2 ), and a 
third period of time (time 3 ). From there, a first t-stat value (t-stat x ) for ave. returni, a second t- 
stat value (t-stat 2 ) for ave. return 2 , and a third t-stat value (t-stat 3 ) for ave. return 3 , are calculated, 
in absolute terms. In particular, the first, second, and third t-stat values are calculated by 
utilizing ave. returni, ave. return 2 , ave. return 3 , a first standard deviation (SDO, a second standard 
deviation (SD 2 ), and a third standard deviation (SD 3 ), a total number of revisions (revisionsi) 
occurring more than or equal to timei before said evaluating, a total number of revisions 
(revisions 2 ) occurring more than or equal to time 2 before said evaluating, and a total number of 
revisions (revisions 3 ) occurring more than or equal to time 3 before said evaluating. 
Subsequently, a first degree of freedom (DFO for ave. returni, a second degree of freedom (DF 2 ) 
for ave. return 2? and a third degree of freedom (DF 3 ) for ave. return 3 are calculated. The 
invention then contemplates determining a first probability (probi) for timei by utilizing t-stati 
and DFi, a second probability (prob 2 ) for time 2 by utilizing t-stat 2 and DF 2 , and a third 
probability (prob 3 ) for time 3 by utilizing t-stat 3 and DF 3 . A raw conditional performance score 
for the analyst is generated by utilizing probi, prob 2 , prob 3 . A conditional performance score 
(cond. performance score) may be generated by adding a modifier (adjustment), which may be 
determined by utilizing revisions i, revisions 2 , and revisions 3 , to the raw conditional performance 
score. From there, the invention contemplates generating a small return penalty by subtracting a 
penalty from an average of return factors corresponding to timei, time 2 , and time 3? wherein for 
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upward revisions the penalty is 0 if ave. returns is greater than 15%, else the penalty is equal to 
15% minus ave. return, wherein for downward revisions the penalty is 0 if ave. returns is less 
than -8%, else said penalty is equal to -8% minus ave. returns, wherein the first period of time 
return factor is ten if ave. returni is greater than ten, else the first period of time return factor is 
equal to a cube root of a product of 100 and ave. returni, wherein the second period of time 
return factor is ten if ave. return^ is greater than ten, else the second period of time return factor is 
equal to a cube root of a product of 100 and ave. return, and wherein the third period of time 
return factor is ten if ave. return 3 is greater than ten, else the third period of time return factor is 
equal to a cube root of a product of 100 and ave. returns. Using the above factors, a final 
10 performance score (performance score) indicative of the analyst's performance may be generated 
by adding the small return penalty to the conditional performance score. 

1% There has thus been outlined, rather broadly, the more important features of the invention 

I U in order that the detailed description thereof that follows may be better understood, and in order 
that the present contribution to the art may be better appreciated. There are, of course, additional 
features of the invention that will be described hereinafter and which will form the subject matter 
? of the claims appended hereto. 

Ji; In this respect, before explaining at least one embodiment of the invention in detail, it is 

2§ to be understood that the invention is not limited in its application to the details of construction 
and to the arrangements of the components set forth in the following description or illustrated in 
the drawings. The invention is capable of other embodiments and of being practiced and carried 
out in various ways. Also, it is to be understood that the phraseology and terminology employed 
herein are for the purpose of description and should not be regarded as limiting. 

As such, those skilled in the art will appreciate that the conception, upon which this 
disclosure is based, may readily be utilized as a basis for the designing of other structures, 
methods and systems for carrying out the several purposes of the present invention. It is 
important, therefore, that the claims be regarded as including such equivalent constructions 
30 insofar as they do not depart from the spirit and scope of the present invention. 
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Further, the purpose of the foregoing abstract is to enable the U.S. Patent and Trademark 
Office and the public generally, and especially the scientists, engineers and practitioners in the 
art who are not familiar with patent or legal terms or phraseology, to determine quickly from a 
cursory inspection the nature and essence of the technical disclosure of the application. The 
abstract is neither intended to define the invention of the application, which is measured by the 
claims, nor is it intended to be limiting as to the scope of the invention in any way. 

These together with other objects of the invention, along with the various features of 
novelty which characterize the invention, are pointed out with particularity in the claims annexed 

10 to and forming a part of this disclosure. For a better understanding of the invention, its operating 
advantages and the specific objects attained by its uses, reference should be had to the 

p accompanying drawings and descriptive matter in which there is illustrated preferred 

1% embodiments of the invention. 



,2 Other objects of the present invention will be evident to those of ordinary skill, 

particularly upon consideration of the following detailed description of the preferred 
s embodiments. 

jl NOTATIONS AND NOMENCLATURE 

10 

The detailed descriptions which follow may be presented in terms of program procedures 
executed on computing or processing systems such as, for example, a stand-alone gaming 
machine, a computer or network of computers. These procedural descriptions and 
representations are the means used by those skilled in the art to most effectively convey the 
substance of their work to others skilled in the art. 

A procedure is here, and generally, conceived to be a self-consistent sequence of steps 
leading to a desired result. These steps are those requiring physical manipulations of physical 
quantities. Usually, though not necessarily, these quantities take the form of electrical or 
30 magnetic signals capable of being stored, transferred, combined, compared and otherwise 

manipulated. It proves convenient at times, principally for reasons of common usage, to refer to 
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these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. It should 
be noted, however, that all of these and similar terms are to be associated with the appropriate 
physical quantities and are merely convenient labels applied to these quantities. 

Further, the manipulations performed are often referred to in terms, such as adding or 
comparing, which are commonly associated with mental operations performed by a human 
operator. No such capability of a human operator is necessary, or desirable in most cases, in any 
of the operations described herein which form part of the present invention; the operations are 
machine operations. Useful machines for performing the operation of the present invention 
include general purpose digital computers or similar devices. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 illustrates a prior art stock and cash portfolio development system; 

FIG. 2 illustrates a prior art securities valuation system; 

FIG. 3 illustrates a prior art investment fund management system; 

FIG. 4 illustrates a prior art data processing system for selecting securities and 
constructing investment portfolios based on a set of artificial neural networks; 

FIGS. 5a-5b depict one example of a user interface utilizable for displaying an analyst 
evaluation according to the principles of the present invention; 

FIGS. 5c-5d depict one example of a user interface displaying examples of revisions 
utilizable in conjunction with the principles of the present invention; 

FIG. 6 is a flow diagram illustrating a process for evaluating analysts according to the 
principles of the present invention; 

FIG. 7 is a flow diagram illustrating a high level process for generating an upward 
revision performance measure utilizable in the process of FIG. 6; 

FIG. 8 is a flow diagram illustrating a high level process for generating a downward 
revision performance measure utilizable in the process of FIG. 6; 

FIG. 9 is a flow diagram illustrating a process for generating an upward revision small 
return adjustment utilizable in the processes of FIG. 7; 
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FIG. 10 is a flow diagram illustrating a process for generating a downward revision small 
return adjustment utilizable in the processes of FIG. 8; 

FIG. 1 1 is a flow diagram illustrating a process for ranking analysts according to an 
upward revision performance measure utilizable in the process of FIG. 7; 

FIG. 12 is a flow diagram illustrating a process for ranking analysts according to a 
downward revision performance measure utilizable in the process of FIG. 8; 

FIG. 13 illustrates one example of a central processing unit for implementing a computer 
process in accordance with a computer implemented embodiment of the present invention; 

FIG. 14 illustrates one example of a block diagram of internal hardware of the central 
1 0 processing unit of FIG. 1 3 ; 

FIG. 15 illustrates another example of a block diagram of internal hardware of the central 
processing unit of FIG. 13; 

FIG. 16 illustrates one example of a memory medium which may be used for storing a 
computer implemented process of the present invention; 
m FIG. 17 illustrates an example of a combined Internet, POTS, and ADSL architecture 

yj which may be used with the present invention; and 

^ FIG. 18 shows a set of tabulated results of an evaluation performed according to the 

C 3 principles of the present invention. 

|| BEST MODE FOR CARRYING OUT THE INVENTION 

Reference now will be made in detail to the presently preferred embodiments of the 
invention. Such embodiments are provided by way of explanation of the invention, which is not 
intended to be limited thereto. In fact, those of ordinary skill in the art may appreciate upon reading 
the present specification and viewing the present drawings that various modifications and variations 
can be made. 

For example, features illustrated or described as part of one embodiment can be used on 
other embodiments to yield a still further embodiment. Additionally, certain features may be 
30 interchanged with similar devices or features not mentioned yet which perform the same or similar 
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functions. It is therefore intended that such modifications and variations are included within the 
totality of the present invention. 

In accordance with the principles of the present invention, an analyst's performance is 
evaluated by utilizing information pertaining to at least one revision made by the analyst 
involving at least one investment. This evaluation includes determining a conditional 
performance score indicative of the analyst's performance relative to other analysts. The 
performance score is determined at least in part by considering an average historical performance 
of the investment following the revision. In addition, the performance score is also determined 
using a historical consistency of the analyst's performances with respect to revisions involving 
the investment, and a number of revisions made by the analyst. Then, the performance score 
may be adjusted according to one or more adjustments to generate a final performance score. 

These performance scores may be determined based on upward or downward revisions or 
a combination of both. In particular, a score based on upward revisions, or an upward revision 
performance score, corresponds to the predictive nature of an analyst with respect to predicting 
or influencing increases in earning. In contrast, a score based on downward revisions, or a 
downward revision performance score, corresponds to the predictive nature of an analyst with 
respect to predicting or influencing decreases in earnings. Similarly, a combined score, or for 
example an average of the two previous scores, corresponds to the overall predictive nature of 
the analyst. 

Referring first to FIGs. 5a and 5b, one example of a user interface 500 utilizable for 
displaying, for example, analyst performance scores is illustrated. In FIG. 5, the user interface 
displays a wide variety of information including performance scores for any number of 
companies 510. To calculate the company performance scores in this example, the scores for 
each of the analysts covering a particular company are averaged to produce an overall company 
score. In addition to listing and determining an average of the performance scores of all the 
analysts covering a company, any combination of performance scores may optionally be 
determined and listed. For instance, an average of only the upward or downward revision 
performance scores for the analysts covering a particular company may be determined and listed. 
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Alternatively, a user may just as easily elect to view a list of only the upward or downward 
revisions for a single company. Similarly, an index of individual analyst performance scores, 
independent of the companies, based either on upward or downward revisions or both may also 
be determined and listed. 

Advantageously, these performance scores are a measure of the predictive nature of each 
analyst's revisions. In one example, a revision is a change in the opinion of an analyst on a 
performance of a company or other entity, for example, overall company revenues, profits, 
market share, and/or how much a company is likely to earn per share in a future quarter, fiscal 
year, or other time period. For instance, as shown in FIGs. 5c-5d, other examples of revisions 
utilizable with embodiments of the present invention include detailed analyst recommendations 
on whether to sell, hold or buy a security 530, earnings surprises 532, estimates submitted 534, 
price/earnings 536, momentum, price/earning ratio, P/E and growth rates, and/or PEG ratio, 
and/or other similar information/estimates. Some causes for an upward revision include: the 
release of a good quarterly or annual earnings report, a positive analysts' meeting or conference 
call with company executives, or improvements at a similar company in the same industry. 
Likewise, causes for a downward revision include: the release of a grim quarterly or annual 
earnings report, a negative analysts' meeting or conference call with company executives, or 
worsening results at a similar company in the same industry. 

In the present invention, the higher the score of the analyst (with one-hundred, for 
example, being the highest), the more predictive on a historical basis. Furthermore, the 
performance score increases with the number of beneficial revisions of the analyst. For instance, 
the performance score increases when a rate of return of an investment increases and the revision 
is an upward revision, or when a rate of return of the investment decreases and the revision is a 
downward revision. Likewise, the performance score decreases when a rate of return of the 
investment decreases and the revision is an upward revision, or when a rate of return of the 
investment increases and the revision is a downward revision. Thus, a high upward revision 
performance score corresponds to historically high positive returns (e.g., the analyst issued an 
upward revision, and the stock went up in value). Similarly, a high downward revision 
performance score corresponds to high negative returns (e.g., the analyst issued a downward 



32 



26119.120 



PATENT 



revision, and the stock went down in value). In addition, it should be noted that although in this 
particular embodiment corporate securities are mentioned as being one example of the analyst's 
revisions, the instant invention is also applicable to and should be construed as encompassing 
any other form of similar investment. Some examples include bonds, currencies, commodities, 
paper, precious metals, minerals, natural resources, mutual funds, and the like or any other 
analogous investments and/or investment entities. 

As will be discussed in greater detail below, several factors are incorporated into these 
performance scores. For instance, an analyst's average return history (e.g., a return amount 
typically in a percentage calculated over a predetermined period of time after the analyst's 
revision) for both upward and downward forecast revisions may initially be considered. Indeed 
in alternative embodiments, this factor is accorded the greatest amount of weight. Next, the 
analyst's historical consistency at issuing upward and downward revisions in a company's stock, 
as measured by, for example, standard deviation and the like are also considered. In a related 
manner, a measure of variability of the analyst's performance may also be considered. Similarly, 
a likelihood that the revision will actually produce an expected result, as embodied by statistical 
characteristics such as probability and the like are also factored into the calculation. Finally, the 
number of revisions that make up an analyst's track record is factored into the performance score 
for the simple reason that the more times an analyst has demonstrated good judgment the better. 
Other factors are optionally used as well. 

As to the analyst's average return history, any number of return periods may be 
considered. For example, in one embodiment, two periods are utilized. In other embodiments, 
three or more periods may be used. Furthermore, any length of time is possible. Thus, in some 
cases, two periods (e.g., a relatively short period of time and a relatively long period of time) 
suffices. In other cases, one or more intermediate periods of time may also be included into the 
calculation. As examples, three month and six month periods may be used. As another example, 
five, twenty, and sixty day periods may be used. Similarly, one week, one month, and two 
month periods are also possible. 
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In addition to the above, other factors may optionally be considered as well. For 
example, the above factors may be utilized to generate a conditional or raw performance score, 
which in turn may be modified or adjusted to produce a final performance score. For instance, a 
modifier accentuating the number of revisions made by the analyst may be factored into the 
conditional performance score. In other cases, a return amount adjustment may be made to 
reward or penalize an analyst for small or large return amounts. For example, the return amount 
adjustment may include a small return penalty penalizing or reducing the analyst's performance 
score for returns less than a predetermined amount. More specifically, a penalty may be imposed 
for returns of less than a predetermined amount as calculated over a predetermined period of 
time. Hence, with upward revisions, a penalty might be imposed for returns of less than 
approximately 15% as calculated over approximately a two month period of time. In contrast, 
with downward revisions, a penalty might be imposed for returns greater than -8% as calculated 
over a two month period of time. Likewise, high returns may also result in increases to the 
analyst's performance score. Further, adjustments need not be limited to return amounts. As an 
example, adjustments may be made to a performance score for an extremely large number of 
accurate trades. Indeed, the present invention contemplates modifications or adjustments for 
nearly any factor utilizable for predicting accuracy or reliability. 

Returning to the interface displayed in FIG. 5, in addition to listing the average scores for 
all of the analysts associated with a particular company, the interface may also be used to list 
each of the highest performing analysts for categories pertaining to downward and upward 
performance scores. Furthermore, interface 500 may also be linked to or used to display 
information concerning each analyst and/or the corresponding security. Likewise, the interface 
sponsor may optionally include commentary concerning, for example, any securities or analysts, 
advertisements utilizable as a source of revenue, instructions or help information for beginning 
users, and information concerning or tracking the activities of exceptionally high performing 
analysts, or other similar features. 

While the interface depicted in FIG. 5 is used to view the average of the performance 
scores associated with a particular list of companies, other listings are also possible. For 
example, a user may select a particular analyst for review. In this case, each of the upward, 



34 



26119.120 



PATENT 



downward, and combined performance scores for that analyst may be viewed. Furthermore, if 
the selected analyst is responsible for monitoring more than one company, multiple values for 
upward, downward, and combined performance scores (e.g., one score for each company) may 
be listed. Similarly, a user may select any combination of analysts or sets/groupings of data 
including, for example, all analysts dealing with or investing in a particular industry or market, 
or other analogous listings of information. 

A broad overview of the process of one embodiment of the present invention for 
calculating performance scores is now illustrated with reference to FIG. 6. First, a list of 
analysts is retrieved from one or any number of databases 610. As will be discussed in greater 
detail below with reference to FIG. 17, the database may include a list of any or all of a variety 
of combinations of analysts. For example, all of the analysts tracking companies in a single 
industry may be included. Similarly, the database may include all insiders filing insider 
documents with the SEC. In other embodiments, other similar or analogous techniques may be 
utilized to generate a list of analysts. For example, any one of a variety of databases tracking 
industry leaders may just as easily be used. 

After retrieving the list of analysts, a smaller list of analysts to be evaluated is optionally 
generated according to predetermined criteria. In this regard, a check is performed identifying 
analyst that meet predetermined criteria 615. In this example, any number of criteria may be 
used to generate this list. For example, each analyst may be required to be an actual living 
person, thereby excluding brokerage firms. Thus, in this scenario, any entities that are not 
people, such as companies, partnerships, or trusts, are removed from the evaluation list. As 
another example, the market capitalization of the company being monitored may be required to 
be greater than a predetermined amount, such as, for instance, $50 million. Third, each analyst 
may be required to have issued a minimum number of revisions that have been in existence for a 
certain period of time. For instance, some embodiments may require analysts to have issued at 
least two revisions, with each revision being at least sixty days old. Other embodiments may 
require more or fewer revisions, which in turn may be more recent or older than sixty days. 
These requirements are disclosed for exemplary purposes only and it is to be understood that 
other types of analyst requirements are possible within the scope of the invention. For instance, 
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non-people analysts such as partnerships, corporations, and other joint ventures or entities are 
also possible where groups of analysts participate in recommendations and/or decision making. 
Thus, a group of, for example, Morgan Stanley analysts may be compared against Goldman- 
Sachs analysts, individually or overall Similarly, entities not having market capitalization or 
trading limits are also possible. Furthermore, the concepts of the present invention are applicable 
to entities investing in investments other than securities, such as bonds, mutual funds, or any 
other similar investments, trading securities, investments and the like. 

If the analyst does not meet the predetermined criteria (615), a message is returned, and 
the analyst is not evaluated 620. On the other hand, if the analyst meets the predetermined 
criteria, processing continues with the calculation of a performance score for the analyst 625. As 
will be discussed below, this performance score may reflect upward revisions, downward 
revisions, or a combination, and/or average distribution of the two. From there, processing 
continues with a determination of whether more analysts exist to be ranked 630. 

/ 

If more analysts are to be ranked, processing returns to 610. If the list of analysts to be 
ranked is complete, each of the analysts meeting the predetermined criteria are ranked 635. 
Subsequently, these rankings may optionally be disseminated 640 via, for example, a user 
interface similar to that depicted in FIG. 5 in a local or global setting, e.g., intranets, the Internet, 
and the like. Alternatively, the rankings may be disseminated in any other similar or analogous 
medium, such as in a magazine or newspaper, a trade journal, television, and radio or the like. 

Although in this embodiment a list of analysts to be evaluated is determined according to 
predetermined criteria, it is to be understood that other alternatives are also possible. For 
instance, the present invention is to be construed as including scenarios where a user enters or 
selects a predetermined listing of analysts for evaluation. In such a case, only the particular 
entities entered by the user would be evaluated. It is also possible that a mixed grouping of 
investment entities and types may be evaluated. For instance, it is possible within the scope of 
the present invention to evaluate corporate insiders, nonperson joint ventures, brokerage firms, 
and/or corporations, and any other types of investment entities or combinations thereof. 
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The calculation of an upward revision performance score is now described with reference 
to FIG. 7. In this example, one or more rates of return on an investment corresponding to an 
analyst's upward revision, taken a predetermined period of time after issuance of the upward 
revision, are utilized in the generation of the performance score. Thus, one or more return values 
for one or more return intervals are initially calculated or gathered for each analyst 705 by any 
suitable means. In this particular embodiment, three return values, one for a short period of time, 
one for an intermediate period of time, and one for a long period of time, are taken. As to the 
actual amount of time associated with each period, any lengths may be used. For example, the 
short period of time may be five days or a week, the intermediate period may be twenty days or a 
month, and the long period may be sixty days or two months. Furthermore, three return values 
are utilized in this embodiment for exemplary purposes only. In actual practice any number of 
return values, at any number of time intervals, are allowed so long as they are suitable proxies 
for, e.g., short, long, and optionally intermediate periods of time. As an example, two return 
values, one at 3 months (e.g., 13 weeks) and a second at 6 months (e.g., 26 weeks), may just as 
easily be utilized. 

In accordance with the principles of the present invention and as discussed briefly above, 
these scores or measures reflect an analyst's performance and are generated by considering a 
number of performance factors. One embodiment of the present invention uses a measure of the 
analyst's historical performance as determined by rates of return over certain periods of time 
after a revision. Another example may include considering the analyst's success at selecting 
entry and exit points in the investment. Additionally, as will be discussed below, the analyst's 
performance score may be modified or weighted based on a number of revisions made by the 
analyst, the magnitude of a return, or other similar factors. Of course, the above factors are listed 
for exemplary purposes only and it is to be understood that other similar and analogous factors 
may also be considered so long as they provide an accurate and/or estimated measurement of the 
analyst's success or lack thereof. 

After calculating the return values, a statistical measure of confidence in the analyst's 
performance may be calculated for each return value 710. As one example, a t-statistic or t-stat, 
taken in absolute terms, is utilized as this measure of confidence. The t-stat is a measure on a 
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random sample (or pair of samples) in which a mean (or pair of means) appears in the numerator, 
and an estimate of the numerator's standard deviation appears in the denominator. The latter 
estimate is based on the calculated sample variance estimates of the samples. Alternatively, 
other standard variance measures or other suitable indices of variability may optionally be used 
in place of standard deviation, and/or t-stat If these calculations yield a value of (t) that is 
sufficiently different from zero, the test is considered to be statistically significant. 



In this embodiment, the t-stats for each return period may be determined by: 

upward revision return 



upward revision t stat = 



upward _ revision _ SD 



<yj total _ number _ of _ upward _ revisions 



In this example since three intervals are being utilized, three t-stat values are calculated, one for 
each interval. 



To prevent division by zero, if the standard deviations for any of the time intervals are 
zero, then the conditional upward revision performance score for that interval is set to zero. In 
addition, although the t-stat is used in this example, other measures of confidence may just as 
easily be incorporated in the present invention. As another example, the z-statistic or other 
similar measurements of variability, dispersion, or consistency may be used. Furthermore, 
regardless of what method is used to produce the measure of confidence, the measure of 
confidence may also be manipulated or weighted to consider other factors, so long as it 
accurately reflects or estimates, for instance, the analyst's return history and historical 
consistency at issuing forecast revisions in the investment. 

After calculating the measures of confidence (e.g., the t-stats for each time interval or 
return period), one or more degrees of freedom are determined by subtracting, for example, one 
from the number of revisions for each return value 715. Hence, with three return values, three 
degrees of freedom are generated. The degrees of freedom are used to describe the number of 
values in a final calculation of a statistic that are free to vary, and measure, for example, 
repetition, repeatability, and certainty. Hence, each degree of freedom is determined by: 
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upward revision degree of freedom = total number of upward revisions - 1 

Next, a probability indicating, for example, the likelihood that an analyst's and/or other 
investing entity's actions will actually produce the expected result, is determined for each return 
interval 720. Thus, with three return intervals, three probabilities are generated. In this example, 
the degrees of freedom, along with the absolute t-stats, are used in determining a probability for 
each return period 720. Specifically, subject to the following optional conditions, the 
probabilities are looked-up in a common statistics lookup table using each return interval's 
degree of freedom and t-stat. If any of the degrees of freedom are greater than 100, then that 
degree of freedom is set to 100 before looking up the probability. If any of the absolute t-stats 
are greater than 20, then that absolute t-stat is set to 20 before looking up the probability. 
Finally, if any of the absolute t-stats are less than 0.01, then the corresponding probability is 
simply set to 0. In addition, the probabilities may be calculated utilizing other methods. For 
instance, the t-stat and the degree of freedom may undergo further manipulation before looking 
up the probability. Alternatively, other analogous methods may be utilized in place of the look- 
up table to determine the probabilities. 

Subsequent to determining the probabilities for each return period, a conditional upward 
revision performance score, or raw conditional upward revision performance score, is generated 
725. Although other methods are possible, the conditional performance score in this example is 
determined by: 

In this formula, up_probl, up_prob2, up_prob3 correspond respectively to the probability for a 
first, second, and third return interval. Referring again to the above formula, whether the 
probabilities are added or subtracted is determined by the following conditions. If any of the 
three probabilities are greater than zero, then that probability is subtracted from 0.5. Otherwise, 
0.5 is added to the probability. 
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The conditional upward revision performance score represents an initial analyst 
performance measure with respect to upward revisions and may undergo one or more of any 
number of manipulations to result in various other refined performance measures and ultimately 
a final upward revision performance score. As an example, the conditional performance score 
may be weighted to place additional emphasis on the maturity of the corporation. Alternatively, 
investments in, for example, conservative sectors may also receive a modification. To more 
clearly illustrate, exemplary modifiers accounting for the size of a return and a number of 
revisions issued are discussed in greater detail below. These modifications are introduced for 
exemplary reasons only and it is to be understood that other manipulations are also possible 
10 within the scope of the present invention. 

Initially, an optional upward revision adjustment may be generated to reflect and 
y emphasize the number of revisions issued by the analyst. Specifically, a modification may be 
Jy made to the conditional upward revision performance score accentuating the number of upward 
% 2 revisions issued by the analyst 730. 



In the above formula, num_up_revl, num_up_rev2, and num_up_rev3 correspond respectively 
20 to the number of upward revisions for a first interval, the number of upward revisions for a 

second interval, and the number of upward revisions for a third interval. As will be noted from 
the above formula, three separate upward revision numbers or values for three time intervals are 
considered in this embodiment. 

As another example, an upward revision return amount adjustment may be generated to 
emphasize a large or small return amount after a revision forecast made by the analyst. This 
adjustment may be utilized to penalize an analyst for a small return, or reward an analyst for a 
large return. Referring to FIG. 9, one example utilizable for determining a return amount 



In this example, the adjustment is determined by: 



up. rev. adj = (-20)* 
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adjustment for an upward revision is illustrated. In this embodiment, the return amount 
adjustment includes a small return penalty for reducing the conditional performance score for 
returns on the investment that are less than a predetermined amount. Although any 
predetermined amount may be utilized, with upward revisions in this example, it is set to 
approximately 15%. The small return penalty is generated using an average of a number of 
return factors and a penalty amount. More particularly, the small return penalty is determined in 
one embodiment by taking a difference between an average of a short period of time return 
factor, an intermediate period of time return factor, and a long period of time return factor, and a 
penalty. Thus, each of a short period of time return factor, an intermediate period of time return 
factor, and a long period of time return factor are initially calculated 905, 910, and 915. 

In this embodiment, subject to certain conditions discussed below, each of the period of 
time return factors (e.g., short, intermediate and long return factors) is equal to a cube root of a 
return percentage over a corresponding period of time, unless the return is greater than 
approximately 10 or 1000%. If the return is greater than approximately 10 or 1000%, the factor 
is simply set to 10 or 1000%. Thus to restate the above, if the average return for the period of 
time is greater than 10, then the return factor is simply set to 10. However, if the average return 
for a period of time is not greater than 10, the return factor is equal to: 

return factor = (100 * average upward revision return) 173 

After calculating each of the return values, the optional penalty is generated 920. In this 
embodiment, the penalty is a difference between a predetermined amount and a return for a 
predetermined period of time. The predetermined period of time is optionally equal to a time 
corresponding to the long period of time return factor. If the average return value corresponding 
to the investment for the predetermined long period of time is greater than a predetermined 
amount, the penalty is simply set to 0. As an example, the period of time and the predetermined 
amount may be approximately sixty days and approximately 0.15 or 15%, respectively. Thus in 
this example, a penalty is calculated only if the return over a sixty day period of time is less than 
15%. However, if the average return value corresponding to the investment for the long period 
of time is not greater than 0. 15 or 15%, the penalty is equal to a product of 100 and a difference 
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between 0.15 (or 15%) and the average return value corresponding to the investment for a sixty 
day period of time. In other words, if the average return value corresponding to the investment 
for the long period of time (e.g., sixty days) is not greater than 0.15 or 15%: 

penalty = (100 * (0.15 - average upward revision return for the long period of time)) 

This penalty is then subtracted from the average of the return factors 925 (discussed 
above) and used to produce an optional adjustment: 

10 adjustment = (short return factor -f intermediate return factor + long return factor)/3 - penalty 

?,« Thus, in this example, a small return penalty is imposed for small returns (e.g., 

U approximately 15%) over a long period of time (e.g., approximately sixty days) after a revision, 
f U 0nce the return factors and penalty have been determined and used to produce an adjustment, the 
"\; adjustment(s) is added to the conditional performance score to produce a final upward revision 
UJ performance score 735. 

H In the above example, adjustments reflecting the number of revisions issued by the 

|£ analyst and reflecting return amounts emphasizing a small return amount are illustrated. 
|Q However, as discussed above, any other modifications are also possible. For instance, 
~f* modifications may be made placing a much greater emphasis on short term investments to 

generate an adjustment aimed at producing scores for use in formulating short term investment 
strategies. Or, in contrast, greater weight may be placed on returns over longer periods of time to 
reflect, for example, a more conservative investment strategy. 

The generation of performance scores based on downward revisions is similar to the 
generation of upward revision performance scores and is depicted in FIG. 8. However, in 
contrast to upward revision performance scores, downward revision performance scores are 
indicative of an analyst's influence or accuracy in predicting the performance of, for example, a 
30 security after issuance of a downward revision. A conditional downward revision performance 
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score may be manipulated in any number of ways to produce refined or final downward revision 
performance scores. 

As with the upward revision scores, processing commences with the calculation or 
determination of one or more return values, or rates of return, for each analyst, taken a 
predetermined period of time after issuance of the downward revision 805. Again, any number 
of return values may be taken. In this particular embodiment, three return values, one for a short 
period of time, one for an intermediate period of time, and one for a long period of time, are 
taken. Furthermore, any lengths may be associated with these periods of time (e.g., a short 
period of time of five days, an intermediate period of twenty days, and a long period sixty days). 
While, three return values are utilized in this embodiment, any number of return values, at any 
number of time intervals, are allowed. 

Subsequently, a statistical measure of confidence in the analyst's performance is 
calculated 810. In this case, a t-statistic (t-stat), in absolute terms, is again utilized for exemplary 
reasons, and is likewise calculated for each return value. Again, since three intervals are utilized 
in this example, three t-stat values are calculated, one for each interval. 

For downward revisions, the t-stats for each return period are determined by: 



downward revision t stat = 



downward revision return 



downward _ revision _ SD ^ 
J total number of downward revisions , 



Like with the upward revision performance score, to prevent division by zero, if the standard 
deviations for any of the three time intervals are zero, then the downward revision performance 
score for that interval is set to zero. 

Again, as with the above example, it is to be understood that other measures of 
confidence may be substituted for the t-stat. Also, any number of modifications are possible. As 
indicated above, as long as an accurate reflection of the analyst's return history and/or historical 
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consistency at issuing downward revisions is considered, any measure of confidence, for 
example a z-stat or other similar measurement of variability, dispersion, or consistency may be 
used. 

After calculating the measures of confidence (e.g., the t-stats for each time interval or 
return period), one or more downward revision degrees of freedom are determined by 
subtracting, for example, one from the number of revisions for each return value 815. Hence, 
with three return values, three degrees of freedom are generated. Each degree of freedom is 
determined by: 

downward revision degree of freedom = total number of downward revisions - 1 

Subsequently, a probability indicating, for example, the likelihood that an analyst's 
and/or other investing entity's actions will actually produce the expected result, is determined for 
each return interval 820. In this example, the degree of freedom values, along with the absolute 
t-stat values, are used in determining a probability for each return period. Specifically, subject to 
the following optional conditions, the probabilities are looked-up in a common statistics lookup 
table using each return interval's degree of freedom and t-stat. If any of the degrees of freedom 
are greater than 100, then that degree of freedom is set to 100 before looking up the probability. 
If any of the absolute t-stats are greater than 20, then that absolute t-stat is set to 20 before 
looking up the probability. Finally, if any of the t-stat are less than .01, then the corresponding 
probability is simply set to 0. Alternatively, the probabilities may be calculated based on other 
the methods. For example, the measures of confidence and degrees of freedom may be modified 
or manipulated before looking up the probability in the lookup table. Also, any other similar 
methods may be used in place of the look-up table to generate a probability. 

Subsequent to determining the probabilities for each return period, a conditional 
downward revision performance score, or raw conditional downward performance score, is 
generated 825. Although other methods are possible, the conditional performance score in this 
example is determined by: 
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cond _ dw _ per _ score = 



[l - (.5 + or - dw_ probl)] + [l - (.5 + or - dw__ probl)] + [l - (.5 + or - dw_prob3)] 

3 



*100 



In this formula, dw_probl, dw_prob2, dw_prob3 correspond respectively to the probability for a 
first, second, and third return interval. Whether the probabilities are added or subtracted is 
determined by the following conditions. In contrast to upward revisions, if any of the three 
probabilities are less than zero, then that probability is subtracted from 0.5. Otherwise, 0.5 is 
added to the probability. 

As with the above, the conditional downward revision performance score represents an 
initial analyst performance measure and may undergo one or more of any number of 
manipulations to result in various other refined performance measures and ultimately a final 
downward revision performance score. As examples, modifiers accounting for the size of return 
and the number of revisions issued are discussed in greater detail below. These modifications 
are introduced for exemplary reasons only and it is to be understood that other manipulations are 
also possible within the scope of the present invention. 

For example, a downward revision adjustment may be generated to reflect and emphasize 
the number of revisions issued by the analyst. Specifically, a modification may be made to the 
conditional downward revision performance score accentuating the number of downward 
revisions issued by the analyst. 

In this example, the adjustment is determined by: 



In the above formula, num_dw_revl, num_dw__rev2, and num„dw_rev3 correspond respectively 
to the number of downward revisions for a first interval, the number of downward revisions for a 
second interval, and the number of downward revisions for a third interval. As will be noted 
from the above formula, three separate downward revision numbers or values for three time 
intervals are considered in this embodiment. 



1 



dw. rev. adj. = (-20) : 



num. _dw_ revl 4- num _ dw_ rev2 + nam _dw_ rev3 



3 
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As another example, a downward revision return amount adjustment may be generated to 
emphasize a large or small return amount after a revision forecast made by the analyst. This 
adjustment may be utilized to penalize an analyst for a small return, or reward an analyst for a 
large return. Referring to FIG. 10, one example utilizable for determining a return amount 
adjustment for a downward revision is illustrated. In this embodiment, the return amount 
adjustment includes a small return penalty for reducing the conditional performance score for 
returns on the investment that are less than a predetermined amount. Although any 
predetermined amount may be utilized, with downward revisions, in this example, it is set to 
approximately negative 8%. The small return penalty is generated using an average of a number 
of return factors and a penalty amount. More particularly, the small return penalty is determined 
by taking a difference between an average of a short period of time return factor, an intermediate 
period of time return factor, and a long period of time return factor, and a penalty. Thus, each of 
a short period of time return factor, an intermediate period of time return factor, and a long 
period of time return factor are initially calculated 1005, 1010, and 1015. 

In this embodiment, subject to certain conditions discussed below, each of the period of 
time return factors (e.g., short, intermediate and long return factors) is equal to a cube root of a 
return percentage over a corresponding period of time, unless the return is greater than 
approximately 10 or 1000%. If the return is greater than approximately 10 or 1000%, the factor 
is simply set to 10 or 1000%. Thus to restate the above, if the average return for the period of 
time is greater than 10, then the return factor is simply set to 10. However, if the average return 
for a period of time is not greater than 10, the return factor is equal to: 

return factor = (100 * average downward revision return) 173 

After calculating each of the return values, the penalty may be generated 1020. In this 
embodiment, the penalty is a difference between a predetermined amount and a return for a 
predetermined period of time. In this example, the predetermined period of time may be equal to 
a time corresponding to the long period of time return factor. If the average return value 
corresponding to the investment for the predetermined period of time is less than a 
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predetermined amount, the penalty is simply set to 0. As an example, the period of time and the 
predetermined amount may be approximately sixty days and approximately negative 0.08 or - 
8%, respectively. Thus in this example, a penalty is calculated only if the return over a sixty day 
period of time is greater than negative 8%. However, if the average return value corresponding 
to the investment for the long period of time is not less than negative 0.08 or -8%, the penalty is 
equal to a product of 100 and a difference between negative 0.08 (or -8%) and the average return 
value corresponding to the investment for a sixty day period of time. In other words, if the 
average return value corresponding to the investment for the long period of time (e.g., sixty days) 
is not less than negative 0.08 or -8%: 

penalty = (100 * (-0.08 - average downward revision return for the long period of time)) 

This penalty is then subtracted from the average of the return factors 1025 (discussed 
above) and used to produce the adjustment: 

adjustment = (short return factor + intermediate return factor + long return factor)/3 - penalty 

Thus, in this example, a small return penalty is imposed for small returns (e.g., 
approximately -8%) over a long period of time (e.g., approximately sixty days) after a revision. 
Once the return factors and penalty have been determined and used to produce an adjustment, the 
adjustment(s) are added to the conditional performance score to produce a final downward 
revision performance score 835. 

In accordance with the principles of the present invention, one example of a process for 
ranking analysts according to upward revision performance score is depicted with reference to 
FIG. 1 1. In particular, this process commences by sorting the analysts into descending (or 
ascending) upward revision performance score order 1105. Subsequently, as will be discussed in 
greater detail below, the analysts are separated into a number of groups 1110. Each group, then, 
is assigned a ranking 1115, with, for example, the group with the highest scores receiving a 
ranking of 100 (signifying the best performance) and the group with the lowest scores receiving 
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a ranking of 0 (signifying the worst performance). Thus, each analyst receives a rank equal to 
his or her group. 

Any method or procedure may be utilized to separate or divide the analysts into groups. 

In one embodiment, the analysts are simply divided to form a standard bell curve distribution. In 
other embodiments, other distributions are possible. As one example, the group of analysts is 
divided into seven groups based on upward revision performance score. 

The first of these seven groups includes the top 5% of the analysts as calculated 
according to upward revision performance score. The analysts falling into this first group are 
then divided into ten substantially equally sized subgroups. The highest of these subgroups is 
assigned a rank of 100. The next subgroup receives a rank of 99. This process continues until 
the last subgroup of the top 5% group is assigned a rank of 91. 

The second group includes the next 12.5% of the analysts as calculated according to 
upward revision performance score (e.g., 5% - 12.5%). The analysts falling into the second 
group are then divided into fifteen substantially equally sized subgroups. The highest of these 
subgroups is assigned a rank of 90. The next subgroup receives a rank of 89. This process 
continues until the last subgroup of the second group is assigned a rank of 76. 

The third group includes the next 18% of the analysts as calculated according to upward 
revision performance score (e.g., 12.5% - 18.0%). The analysts falling into the third group are 
divided into fifteen substantially equally sized subgroups. The highest of these subgroups is then 
assigned a rank of 75. The next subgroup receives a rank of 74. This process continues until the 
last subgroup of the second group is assigned a rank of 61. 

The fourth group includes the next 29% of the analysts as calculated according to upward 
revision performance score (e.g., 18% - 64.5%). The analysts falling into the fourth group are 
divided into twenty substantially equally sized subgroups. The highest of these subgroups is 
then assigned a rank of 60. The next subgroup receives a rank of 59. This process continues 
until the last subgroup of the second group is assigned a rank of 41. 
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The fifth group includes the next 18% of the analysts as calculated according to upward 
revision performance score (e.g., 64.5% - 82.5%). The analysts falling into the fifth group are 
then divided into fifteen substantially equally sized subgroups. The highest of these subgroups is 
assigned a rank of 40. The next subgroup receives a rank of 39. This process continues until the 
last subgroup of the second group is assigned a rank of 26. 

The sixth group includes the next 12.5% of the analysts as calculated according to 
upward revision performance score (e.g., 82.5% - 95%). The analysts falling into the sixth group 
are divided into fifteen substantially equally sized subgroups. The highest of these subgroups is 
then assigned a rank of 25. The next subgroup receives a rank of 24. This process continues 
until the last subgroup of the second group is assigned a rank of 1 1 . 

The seventh or last group includes the last 5% of the analysts as calculated according to 
upward revision performance score (e.g., 95% - 100%). The analysts falling into the seventh 
group are then divided into ten substantially equally sized subgroups. The highest of these 
subgroups is then assigned a rank of 10. The next subgroup receives a rank of 9. This process 
continues until the last subgroup of the second group is assigned a rank of 1. 

As depicted in FIG. 12, a similar procedure is utilized to rank the analysts with respect to 
downward revision performance scores. Thus this process is only briefly summarized here. 
Initially, the process commences by sorting the analysts into descending (or ascending) 
downward revision performance score order 1205. Subsequently, the analysts are separated into 
a number of groups 1210. Then each group is assigned a ranking 1215, with, for example, the 
group with the highest scores receiving a ranking of 100 (signifying the best performance) and 
the group with the lowest scores receiving a ranking of 0 (signifying the worst performance). 
Like with the upward revision scores, each analyst receives a rank equal to his or her group. 

Similar to the upward revision procedure used to separate the analysts, any method may 
be utilized to divide these sorted analysts into groups. As one example, the group of analysts is 
divided into seven groups based on downward revision performance score. For instance, the first 
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group may correspond to the top 5% and receive ranks ranging from 100-91. The second group 
may correspond to the next 12.5% of the analysts and receive ranks ranging from 90-76. The 
third group may correspond to the next 18% of the analysts and receive ranks ranging from 75- 
61. The fourth may correspond to the next 29% of the analysts and receive ranks ranging from 
60- 41. The fifth may correspond to the next 18% of the analysts and receive ranks ranging from 
40-26. The sixth group may correspond to the next 12.5% of the analysts and receive ranks 
ranging from 25-1 1. The seventh or last group may correspond to the last 5% of the analysts and 
receive ranks ranging from 10-1. 

In addition, as mentioned above, any other similar and analogous methods may be used to 
separate and/or divide the analysts into groups. For instance, each group may be required to 
include an equal number of analysts. Similarly, different criteria may be used to determine the 
groups. 

One example depicting the tabulated results of processing in accordance with the above 
techniques with respect to a group of analysts covering a single company (IBM) is shown in FIG. 
18. In this example, both upward and downward performance scores are listed. Also, as 
discussed above, the scores may be averaged or otherwise combined to yield a combined 
performance score. Other alternatives are also possible. 

Viewed externally in FIG. 13, a computer system designated by reference numeral 140 
has a computer 142 having disk drives 144 and 146. Disk drive indications 144 and 146 are 
merely symbolic of a number of disk drives which might be accommodated by the computer 
system. Typically, these would include a floppy disk drive 144, a hard disk drive (not shown 
externally) and a CD ROM indicated by slot 146. The number and type of drives vary, typically 
with different computer configurations. Disk drives 144 and 146 are in fact optional, and for 
space considerations, are easily omitted from the computer system used in conjunction with the 
production process/apparatus described herein. 

The computer system also has an optional display upon which information, such as the 
screen illustrated in FIG. 5 may be displayed. In some situations, a keyboard 150 and a mouse 
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152 are provided as input devices through which a user's actions may be inputted, thus allowing 
input to interface with the central processing unit 142. Then again, for enhanced portability, the 
keyboard 150 is either a limited function keyboard or omitted in its entirety. In addition, mouse 
152 optionally is a touch pad control device, or a track ball device, or even omitted in its entirety 
as well, and similarly may be used to input a user's selections. In addition, the computer system 
also optionally includes at least one infrared transmitter and/or infrared received for either 
transmitting and/or receiving infrared signals, as described below. 

FIG. 14 illustrates a block diagram of the internal hardware of the computer system 140 
of FIG. 13. A bus 156 serves as the main information highway interconnecting the other 
components of the computer system 140. CPU 158 is the central processing unit of the system, 
performing calculations and logic operations required to execute the processes of the instant 
invention as well as other programs. Read only memory (ROM) 160 and random access memory 
(RAM) 162 constitute the main memory of the computer. Disk controller 164 interfaces one or 
more disk drives to the system bus 156. These disk drives are, for example, floppy disk drives 
such as 170, or CD ROM or DVD (digital video disks) drive such as 166, or internal or external 
hard drives 168. As indicated previously, these various disk drives and disk controllers are 
optional devices. 

A display interface 172 interfaces display 148 and permits information from the bus 156 
to be displayed on the display 148. Again as indicated, display 148 is also an optional accessory. 
For example, display 148 could be substituted or omitted. Communications with external 
devices, for example, the other components of the system described herein, occur utilizing 
communication port 174. For example, optical fibers and/or electrical cables and/or conductors 
and/or optical communication (e.g., infrared, and the like) and/or wireless communication (e.g., 
radio frequency (RF), and the like) can be used as the transport medium between the external 
devices and communication port 174. Peripheral interface 154 interfaces the keyboard 150 and 
the mouse 152, permitting input data to be transmitted to the bus 156. In addition to the standard 
components of the computer, the computer also optionally includes an infrared transmitter 178 
and/or infrared receiver 176. Infrared transmitters are optionally utilized when the computer 
system is used in conjunction with one or more of the processing components/stations that 
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transmits/receives data via infrared signal transmission. Instead of utilizing an infrared 
transmitter or infrared receiver, the computer system may also optionally use a low power radio 
transmitter 180 and/or a low power radio receiver 182 as shown in the alternate embodiment of 
FIG. 15. The low power radio transmitter transmits the signal for reception by components of 
the production process, and receives signals from the components via the low power radio 
receiver. The low power radio transmitter and/or receiver are standard devices in industry. 

FIGS. 16 is an illustration of an exemplary memory medium 184 which can be used with 
disk drives illustrated in FIGS. 14-15. Typically, memory media such as floppy disks, or a CD 
ROM, or a digital video disk will contain, for example, a multi-byte locale for a single byte 
language and the program information for controlling the computer to enable the computer to 
perform the functions described herein. Alternatively, ROM 160 and/or RAM 162 illustrated in 
FIGS. 14-15 can also be used to store the program information that is used to instruct the central 
processing unit 158 to perform the operations associated with the instant processes. 

Although computer system 140 is illustrated having a single processor, a single hard disk 
drive and a single local memory, the system 140 is optionally suitably equipped with any 
multitude or combination of processors or storage devices. Computer system 140 is, in point of 
fact, able to be replaced by, or combined with, any suitable processing system operative in 
accordance with the principles of the present invention, including sophisticated calculators, and 
hand-held, laptop/notebook, mini, mainframe and super computers, as well as processing system 
network combinations of the same. 

Conventional processing system architecture is more fully discussed in Computer 
Organization and Architecture , by William Stallings, MacMillan Publishing Co. (3rd ed. 1993); 
conventional processing system network design is more fully discussed in Data Network Design , 
by Darren L. Spohn, McGraw-Hill, Inc. (1993), and conventional data communications are more 
fully discussed in Data Communications Principles , by R.D. Gitlin, J.F. Hayes and S.B. 
Weinstain, Plenum Press (1992) and in The Irwin Handbook of Telecommunications , by James 
Harry Green, Irwin Professional Publishing (2nd ed. 1992). Each of the foregoing publications 
is incorporated herein by reference. Alternatively, the hardware configuration is, for example, 
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arranged according to the multiple instruction multiple data (MIMD) multiprocessor format for 
additional computing efficiency. The details of this form of computer architecture are disclosed 
in greater detail in, for example, U.S. Patent No. 5,163,131; Boxer, A., Where Buses Cannot Go, 
IEEE Spectrum, February 1995, pp. 41-45; and Barroso, L.A. et al, RPM: A Rapid Prototyping 
Engine for Multiprocessor Systems, IEEE Computer February 1995, pp. 26-34, all of which are 
incorporated herein by reference. 

In alternate preferred embodiments, the above-identified processor, and, in particular, 
CPU 158, may be replaced by or combined with any other suitable processing circuits, including 
programmable logic devices, such as PALs (programmable array logic) and PLAs 
(programmable logic arrays). DSPs (digital signal processors), FPGAs (field programmable gate 
arrays), ASICs (application specific integrated circuits), VLSIs (very large scale integrated 
circuits) or the like. 

FIG. 17 is an illustration of the architecture of the combined Internet, POTS (plain, old, 
telephone service), and ADSL (asymmetric, digital, subscriber line) for use in accordance with 
the principles of the present invention. Furthermore, it is to be understood that the use of the 
Internet, ADSL, and POTS are for exemplary reasons only and that any suitable communications 
network may be substituted without departing from the principles of the present invention. This 
particular example is briefly discussed below. 

In FIG. 17, to preserve POTS and to prevent a fault in the ADSL equipment 254, 256 
from compromising analog voice traffic 226, 296 the voice part of the spectrum (the lowest 4 
kHz) is separated from the rest by a passive filter, called a POTS splitter 258, 260. The rest of 
the available bandwidth - - from about 10 kHz to 1 MHz - - carries data at rates up to 6 bits per 
second for every hertz of bandwidth from data equipment 262, 264, and 294. The ADSL 
equipment 256 then has access to a number of destinations including significantly the Internet 
220 or other data communications networks, and other destinations 270, 272. 

To exploit the higher frequencies, ADSL makes use of advanced modulation techniques, 
of which the best known is the discrete multitone (DMT) technology. As its name implies, 
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ADSL transmits data asymmetrically - - at different rates upstream toward the central office 252 
and downstream toward the subscriber 250. 

Cable television providers are providing analogous Internet service to PC users over their 
TV cable systems by means of special cable modems. Such modems are capable of transmitting 
up to 30 Mb/s over hybrid fiber/coax system, which use fiber to bring signals to a neighborhood 
and coax to distribute it to individual subscribers. 

Cable modems come in many forms. Most create a downstream data stream out of one of 
the 6-MHz TV channels that occupy spectrum above 50 MHz (and more likely 550 MHz) and 
carve an upstream channel out of the 5-50-MHz band, which is currently unused. Using 64-state 
quadrature amplitude modulation (64 QAM), a downstream channel can realistically transmit 
about 30 Mb/s (the oft-quoted lower speed of 10 Mb/s refers to PC rates associated with Ethernet 
connections). Upstream rates differ considerably from vendor to vendor, but good hybrid 
fiber/coax systems can deliver upstream speeds of a few megabits per second. Thus, like ADSL, 
cable modems transmit much more information downstream than upstream. Then Internet 
architecture 220 and ADSL architecture 254, 256 may also be combined with, for example, user 
networks 222, 224, and 228. 

In accordance with the principles of the present invention, in one example, a main 
computing server implementing the process of the invention may be located on one computing 
node or terminal (e.g., on user network 222, or system 240). Then, various users may interface 
with the main server via, for instance, the ADSL equipment discussed above, and access the 
information and processes of the present invention from remotely located PCs. More 
specifically, a process 242 capable of performing the analyst evaluating procedure of the instant 
invention may be advantageously implemented in system 240. As depicted in FIG. 17, process 
242 may access a list of analysts stored, for instance in data structure 294. Furthermore, 
although data structure 294 is shown as being stored in system 240, it may just as easily be 
located in a remote and distinct computer system. For example, process 242 may be 
implemented in such a manner as to have access to one of any number of databases 281, 282, 
283,285, 286, 287 or 288. 
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Furthermore, the analyst evaluating process of the present invention may also be 
implemented manually. For instance, it is possible to evaluate analysts by hand without the 
assistance of computing systems. 

The many features and advantages of the invention are apparent from the detailed 
specification, and thus, it is intended by the appended claims to cover all such features and 
advantages of the invention which fall within the true spirit and scope of the invention. Further, 
since numerous modifications and variations will readily occur to those skilled in the art, it is not 
desired to limit the invention to the exact construction and operation illustrated and described, 
and accordingly, all suitable modifications and equivalents may be resorted to, falling within the 
scope of the invention. 
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